AO-A060  849  HONEYWELL  INC  MINNEAPOLIS  MINN  SYSTEMS  AND  RESEARCH  —ETC  F/6  17/5 
PROTOTYPE  AUTOMATIC  TARGET  SCREENER.(U) 

JUN  78  0 E SOLANO*  P M NARENORA*  R C FITCH 

78SRC54-2 


UNCLASSIFIED 


DAAK70-77-C-024e 

NL 


DDC  FILE.CpPil  ADA0  60  8 49 


PROTOTYPE  AUTOMATIC  TARGET  SCREENER 


By 

D.E.  Soland 
P.M.  Narendra 
R.C.  Fitch 
D.V.  Serreyn 
T.G.  Kopet 

Honeywell 

SYSTEMS  & RESEARCH  CENTER 

2600  RI06WAY  PARKWAY 
MINNEAPOLIS,  MINNESOTA  55413 

June  15,  1978 


D D e ' 

fj)Eii32nrii22p?! 

W NOV  6 l»M  j 

iitEEHEiririP-! 

A 


Quarterly  Progress  Report  1 January-31  March  1978 


APPROVED  FOR  PUBLIC  RELEASE; 
DISTRIBUTION  UNLIMITED. 


Prepared  for 

U.S.  Army  Mohility  Equipment 
Research  and  Development  Command, 
Night  Vision  and  Electro-Optics  Lahoratory 
Fort  Belvoir,  Virginia  22060 

10  Hu 


"The  views,  opinions,  and/or  findings  contained  in  this 
report  are  those  of  the  authors  and  should  not  be  construed 
as  an  official  Department  of  the  Army  position,  policy,  or 
decision,  imless  so  designated  by  other  documentations.  " 


HITY  Ct-ASSIHCATtON  Of  THIS  PAGt  (VUMtN  OATA  tNTERfcOI 


READ  INSTRUCTIONS 
BEFORE  COMPLETING  FORM 


REPORT  DOCUMENTATION  PAGE 


a.  QOVT  ACCCSSION  NUMBCH 


iCNi  •%  catalog  number 


I NUMBER 


CJiiart^ii^N^i'dgress  Rv 

1 JaiMBBiX'^  :U  MuiMk 


Al'TdMAT'Ic:  'rAHGKT'  SCKKIINKII 


.NT  NUMBeH(S) 


J.TASK  AMtA 


JO.  PROQMAW  ELf.MtNT.PI 


iganiza; 


1 \ CONT  MOLLINO  OPFiCC  NAMC/AOURtSS 


uui  KUH'lt'o-Oplir 
Virginia  220G0 


14 ’ monitoring  AOENCV  n. 


154,  DEtUASMFICATlON  DOWNQHADINa  SCMEDULl 


16  OISI  HIBUTION  STATEMENT  (OF  THIS  REPORT) 


;■  UISI  HIBUIION  STATEMtNT  (OF  THE  ABSTRACT  tNTtPED  IN  BLOCK  *0,  IF  DIFFERENT  FROM  RfePoRT) 


III  •.ul’t’lEMENTARV  NOTES 


T’ai’gE't  t'ocdgnition 
Pat  lorn  recognition 
linage  pi'oee.s.sing 
Heal-lime 


Infrared 

I'l.lH 

'I'argt'l  eueing 
Target  sereening 


.\BSTAaCT  (CONTINuI 


^Thi.s  report  i.s  the  .second  pnarterly  progres.s  report  for  contract  I).AAK70-77-C-02-lit, 
Prototype  Avitonaatic  'Target  Screener.  It  de.scribe.s  re.sults  of  a five-month  design 
study.  'The  objective  of  the  effort  is  an  automatic  target  screener  to  be  used  with 
ihc'rmnl  imaging  systems  employing  common  module  compoiu'iits.  'This  report  covers 
the  period  from  1 .January  to  dl  March  1078.  i 


EDITION  OF  t NOV  SS  IS  OBSOLETE 


PNCi..A.ssiPii:i) 

SECURITY  CLASSIFICATION  OF  THIS  PAGE  (WHEN  DATA  ENII  HL^ 


lUltf 


CONTENTS 


Section 

1 

0 


•Jf’l 

o 


in  i-.: 


INTRODUCTION  AND  SUMMARY 


Page 

1 


IMAGE  ENHANCEMENT  4 

Synthetic  DC  Restoration  4 

Global  Gain/Rias  Control  12 

Global  Gain /Bias  Control 

Implementation  and  Software  12 

Global  Gain/Bias  Testing  16 

Local  Area  Gain/Brightness  Control  16 


TARGET  SCREENING  ALGORITHM  SIMULATION  17 

Summary  1 7 

Data  Base  Preparation  18 

Image  Segmentation  18 

Thresholds  19 

Threshold  Selection  19 

Interval  Generation  21 

Results  of  Segmentation  of  NVL  Data  Base  23 

Feature  Extraction  and  Evaluation  34 

Recognition  Classifier  37 

Interframe  Analysis  43 


iii 


i 


CONTENTS  (concluded) 


Section  Page 

4 SYSTEM  DESIGN  50 

Hardware  Design  50 

Autothreshold  52 

Interval  Generation  60 

First  Level  Features  64 

Compmter  System  1 66 

Computer  System  2 71 

Symbol  Generation  71 

Timing  and  Synchronization  75 

Physical  Configuration  75 

Software  Design  79 

Data  Acquisition  79 

Data  Processing  81 

Computational  Requirements  90 

CPU  2 Functions  94 

Interframe  Analysis  and  Symbol  Generation  95 

Diagnostics,  Feature  Dumps,  and  Test  Data 
Generation  96 

f 

CPU  1 Software  Development  97 

CPU  2 Software  Development  97 

5 PLANS  FOR  THE  NEXT  REPORTING  PERIOD  98 

REFERENCES  99 


iv 


I 

1 


Fiffure  Page 

1 Functional  Image  Enhancement  System  on  PATS  5 

2a  Light- Emitting  Diode  Overlap  with  2 ;1  Interlace  8 

2b  Normalized  Luminance  Intensity  Distribution 

for  Separate  Fields  ( Odd, Even)  8 

2c  Cumulative  Normalized  Luminous  Intensity 

Distribution  8 

3a  Sample  Thermal  Scene  and  Detector  Scan  10 

3b  Single  Detector  Output  to  LED  Driver  for 

Consecutive  Fields  10 

4a  Vidicon  Scanning  of  LED  Sweep  11 

4b  Vidicon  Video  Format  1 1 

5 Global  Gain /Bias  Control  Unit  13 

6 Global  Gain /Bias  Control  Software  Flowchart  14 

7 Sample  Digitized  FLIR  Imagery  from  tlie  NVL 

Videotape  20 

8 Result  of  One- Directional  Interval  Generation 

Criterion  22 

9 Symmetric  Interval  Generation  Criterion  24 

10  Result  of  Using  Symmetric  Criterion  for  Interval 

Generation  25 

11  Results  of  a Scene  Adaptive  Segmentation  Scheme  26-27 


r 


V 


LIST  OF  ILLUSTRATIONS  (continued) 


Figure  Page 

12  Examples  of  Object  Extraction  28 

13  Examples  of  Object  Extraction  29 

14  Examples  of  Object  Extraction  30 

15  Examples  of  Object  Extraction  31 

16  Ground  Truth  Distribution  of  Segmented  Objects  33 

17  PATS  Cascaded  Thresholded  Classifier  Approach  36 

18  Fourier  Amplitude  Features  (a^,  a^) 

(Show  no  separation  between  target^ 

classes  A,  B,  C,  D)  39 

19  Fourier  Amplitude  Features  (a^,  a^) 

(Show  no  separation  between  target 

classes  A,  B,  C,  D)  40 

20  Scatter  Plot  of  ju20  (intensity)  vs.  /j20  (silhouette) 

(Shows  correlation  of  "like"  moments  from 

different  moment  classes)  41 

21  An  Example  of  a Sequence  of  Tactical  Imagery  45-46 

22  Result  of  Segmentation  of  FLIR  Frame  Sequence 

for  Interframe  Analysis  47 

23  PATS  Functional  Units  51 

24  Autothreshold  Structure  53 

2 5 Edge  Signal  Derivation  54 

vi 


LIST  OF  ILLUSTRATIONS  (continued) 


Figure 

Page 

26 

Adaptive  Edge  Threshold 

55 

27 

Background  Estimator 

57 

28 

Background  Filter  and  Switch  Rationale 

58 

28 

Adaptive  Brig)\t  Threshold 

59 

30 

Interval  Generation 

61 

31 

Start /Stop  Criteria 

62 

32 

Validation  Criteria 

63 

33 

First  Level  Features 

65 

34 

Background  Feature 

67 

35 

Intensity  Feature 

68 

36 

Video  Field  Memory  Bit  Plane 

69 

37 

Computer  System  1 

70 

38 

Memory  1 Configuration /Interface 

72 

39 

CPU  2 Configuration 

73 

40 

Symbol  Generation 

74 

41 

Timing  and  Sync 

76 

42 

PATS  Physical  Configuration 

77 

43 

PATS  Pliyslcal  Layout 

78 

vil 


LIST  OF  ILLUSTRATIONS  (concluded) 


i 

\ 

i 

» 

I 

t 


Flffure 

Page 

44 

Software  Functions 

80 

45 

Interval  Data  Format 

82 

46 

Bin  Matching  Software 

83 

47 

Map  of  Interval  Memory 

85 

48 

Median  Filter  Software 

88 

LIST  OF  TABLES 


Summary  of  the  Digitized  FLIR  Data  Base 
(NVL  Data) 

Clutter  Discrimination  Features 

Clutter  Rejection  Performance 

Recognition  Features  Used  in  Prototype  Classifier 

Target  Class  Definition  for  the  Recognition 
Classifier 

Performance  on  Training  Data 

Object  Classification  for  Sequence  from  Figure  22 

Possible  Bin  Format 

Preliminary  Features  and  Clutter  Prescreening 
Primary  Features  and  Classifier 
CPU  1 Worst  Case  Computational  Requirements 
CPU  2 Software  Functions 


Symbol  Generation  Software 


SECTION  1 


INTRODUCTION  AND  SUMMARY 

This  is  the  second  quarterly  technical  progress  report  for  Contract  No. 
DAAK70-77-C-0248,  Prototype  Automatic  Target  Screener  (PATS).  The 
report  describes  results  of  the  second  half  of  a five-month  Phase  I 
design  study  for  an  automatic  target  screener  that  can  operate  with  first 
generation  thermal  imagers  employing  common  module  components.  The 
period  covered  by  this  report  is  1 January  to  31  March  1978. 

The  objective  of  this  effort  is  to  produce  a design  for  a prototype  automatic 
target  screener  (PATS).  The  screener  will  reduce  the  task  loading  on  the 
thermal  imager  operator  by  detecting  and  recognizing  a limited  set  of  high 
priority  targets  at  ranges  comparable  to  or  greater  than  those  for  an  un- 
assisted observer.  A second  objective  is  to  provide  enhancement  of  the 
video  presentation  to  the  operator.  The  image  enhancement  includes;  1) 
automatic  gain/brightness  control,  to  relieve  the  operator  of  the  necessity 
to  continually  adjust  the  display  gain  and  brightness  controls;  and  2)  DC 
restoration,  to  eliminate  artifacts  resulting  from  AC  coupling  of  the  infra- 
red (IR)  detectors. 

The  report  consists  of  four  principal  sections.  Section  2 describes  the 
effort  under  the  image  enhancement  part  of  the  study;  Section  3 describes 
the  target  screener  design  activities.  Section  4 describes  the  results  of 
the  preliminary  system  design  task.  Plans  for  the  next  three-r'onth 
reporting  period  are  included  in  Section  5. 
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The  image  enhancement  portion  of  PATS  will  consist  of  circuitry  to  operate 
on  the  Common  Module  F'LIR  (MODFLIR)  video  output  signal.  This  circuitry 
will  provide  global  gain  and  bias  control  in  the  form  of  feedback  to  the 
MODFLIR  to  maintain  the  signal  within  the  dynamic  range  of  the  electro- 
optical  multiplexer.  The  global  gain  and  bias  control  circuit  preliminary 
design  has  been  completed  and  will  be  implemented  upon  receipt  of  the  GFK 
MODFLIR. 

Image  enhancement  will  also  include  local  area  gain  and  brightness  control 
to  enhance  local  variations  of  contrast  and  compress  the  overall  scene 
dynamic  range  to  match  that  of  the  display.  This  circuitry  has  been  com- 
pleted and  examples  of  its  performance  on  videotaped  thermal  image  data 
were  included,  along  with  the  circuit  description,  in  the  prev'ious  quarterly 
report? 

The  third  image  enhancement  circuit  is  for  DC  restoration,  to  eliminate  the 
streaking  associated  with  loss  of  line-to-line  correlation  on  the  displayed 
image  because  of  the  AC  coupling  of  the  detector  channels.  A breadboard 
version  of  the  DC  restore  concept  previously  described  has  been  designed 
and  constructed  and  is  undergoing  checkout. 

Sections  3 and  4 summarize  the  results  of  the  target  screener  design  task. 
Section  3 describes  the  results  of  the  subtasks  involved  with  the  design  of 
algorithms  for  detecting  and  recognizing  targets.  Section  4 describes  the 
system  design  preliminary  results. 

^D.  E,  Soland,  et  al. , "Prototype  Automatic  Target  Screener,  " Quarterly 
Progress  Report,  Contract  No.  DAAK70-77-C-0248,  ADA  050684, 

January  15,  1978. 
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The  target  screening  activities  including  data  preparation  and  analysis, 
image  segmentation,  and  feature  extraction  were  reported  previously?' 

The  object  classification  and  target  decision  tasks  remaining  are  describ- 
ed in  Section  3.  A total  of  385  frames  of  FLIR  imagery  has  been  digitized, 
annotated,  and  debanded  where  necessary.  These  images  contain  tanks, 
armored  personnel  carriers  (APCs),  and  some  trucks,  and  represent 
most  of  the  imagery  required  for  target  screener  design,  training,  and 
testing  for  the  tank  and  APC  classes. 
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I'liis  st'(iii)n  ri'pi>rls  llu'  propross  on  tho  ininpo  onhnncoinont  tnsks  of  the 
l’r(notypo  Auloinntu'  ’I’ai'pol  Sn  oonor  (TATS).  Spocificnlly,  tho  tasks 
aiKh  i'ssc'ii  ai'o  synlholii'  DC'  ri'Sii^ration,  plobal  pain  aiui  bias  ronli'ol,  ami 
loial  aroa  pain  ami  bi  ipiPnoss  i'oi\lrol. 

l’'ipui'o  1 is  a functional  iliapi'ain  showinp  those  Ihi'oo  functions  in  I’A'l'S. 

The  haialwaro  for  the  all-aiK\lop  synthetic  IK"  restcu’at  ion  has  been  breml- 
boai'deii  ami  is  beinp  tested;  propress  will  be  presented,  I'he  plobal  pain/ 
bias  control  hardwaia'  has  been  built  and  one  alporithm  is  beinp  testeil  on 
tlu'  FI, 111;  this  alporithni  will  be  discussed.  New  di'velopnu'iits  on  local 
area  pain/briphtness  control  will  be  piven. 

SVN'niK’l'ir  IK"  lli:S  TlKlATlON 

riu'  alporitluu,  iniplenienlation,  and  iustificalion  for  the  all-analop  approacli 
to  syntlu'tic  IK’  i-estoration  wert'  discussed  prt'vaously!  .Sinc»'  then,  the 
hardware  has  bi'en  bnaulboa rdeil  and  is  currently  beinp  testial  on  thi'  simu- 
lation lest  patterns.  The  results  from  these  test  patti'rns  havt*  biam  poi>d 
thus  far.  'I'estinp  on  the  Mc''!)!-"! ,111  is  startinp. 

An  earlier  report'  did  not  cover  fully  all  of  the  assumptions  for  the  analop 
synthetic  IK"  ri'sloration  schenu'.  I'liesi'  assumptii>ns  are  discussed  below, 
and  those  rc'lnti'd  to  specific  common  module  l''I,lll  properties  are  covered 
in  detail. 
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'Fhe  major  assumptions  for  synlfietio  DC  restoration  are; 

1.  The  averaffe  value  of  any  video  line  is  constant  over  tiie  scene. 

2.  The  actual  scene  background  is  varyinf^  slowly  in  the  vertical 
direction. 

3.  The  scene  background  occupies  more  than  50  percent  of  a scan 
line  of  video. 


I 


k ;{ 

hi 


1 


4.  A straight  line  in  the  scene  is  straight  in  the  video  from  the  FLIR. 
(There  are  no  major  vidicon  scanning  nonlinearities. ) 

5.  The  video  from  the  FLIR  is  not  rotated  with  respect  to  the  scene, 
or  more  specifically,  with  respect  to  the  detector  array.  (Tliere 
is  no  vidicon  rotation.) 

6.  Any  liorizontal  scan  line  from  tin  vidicon  is  produced  by  a single 
detector  or  by  weiglited  sums  of  adjacent  detectors  in  the  FLIR. 

Assumptions  1 to  3 arc  scene-dependent.  Assumptions  4 and  5 are  proper- 
ties of  the  vidicon,  and  Assurrjption  6 is  related  to  the  I.KD-vidicon  interface 
and  tlie  scanning  systems  on  both  sides  of  the  interfaces.  The  following 
discussion  gives  the  i-easons  for  Assumptions  1 to  3 and  shows  tliat 
Assumptions  4 to  6 can  be  satisfied  in  the  contmon  module  FLIR. 

Assumption  1 ensures  that  the  average  difference  between  two  consecutive 
video  lines  is  zero.  Assumption  2 witli  Assumption  1 ensures  that  the 
difference  is  constant  Ix'tween  consecutivt*  video  lines  in  tiie  background 
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I’t'jjions  ot  those  lines.  Witli  Assumption  3,  thresholding  will  provide  the 

DC?  shift  value  of  the  difference  between  background  regions;  tliis  technique 

0 

was  discussed  in  an  earlier  report.*" 

Adjustments  are  available  in  tiie  M('>DFMR  to  adjust  edge  straightness 

O 

(Assumption  2)  and  vidicon  rotation  (Assumption  3).  Tiie  accuracies  of 
these  adjustments  are  not  specified  but  they  should  be  more  than  adequate 
for  DC?  restore  to  work  properly;  small  overlap  on  tiie  ends  of  a video 
litu'  will  not  affect  thresholding  or  averages  significantly.  If  this  is  a 
proiilem,  the  ends  can  t>e  "cut  off"  to  ignore  information  in  these  regions. 

To  determine  the  validity  of  Assumption  6,  the  common  module  FUR  must 
be  considered.  In  the  FUR,  the  thermal  scene  is  painted  on  Uie  vidicon 
surface  by  a linear  array  of  up  to  80  liglU -emitting  diodes  (LED)  scanned 
horizontally  in  a 2:1  interlace  format.  Each  detector  output  drives  one 
LED.  The  scanning  format  and  the  LED  array  dimensions  are  shown  in 
Figure  2a.  Hecause  the  luminous  intensity  is  not  uniform  over  the  entire 
LED  surface,  a scene  painted  in  one  field  is  more  intense  at  the  LED 
center  ttian  at  the  LED's  outer  edges.  Ttie  2:1  interlace  scanning  overlaps 
in  such  a way  that  LED  centers  in  one  field  scan  over  the  gaps  in  the 
LEDs  in  the  other  field.  The  approximate,  normalized  luminous  intensity 


2 

Soland,  p.  17. 

3 

Texas  Instruments  Inc.,  "Operation  and  Maintenance  Instructions  with 
Parts  l.ist  for  Multipurpose  Observation  Device,"  September,  1976. 


7 


distribution  (NLID)  is  sliown  for  individual  fields  in  Figure  lb  and  for  fields 
taken  together  in  Figure  Ic.*^  The  accumulative  NLID  gives  a close-to- 
uniform  "painted"  scene  on  the  vidicon  with  additive  overlap  between 
consecutive  LED  centers  in  adjacent  fields.  The  resulting  region  between 
LED  centers  is  an  interpolation  between  adjacent  LEDs. 

A line  of  video  read  from  the  vidicon  should  correspond  to  the  output  of  a 
single  LED.  This  assumption  is  extended  to  include  the  weighted  sum  of 
consecutive  LEDs  as  is  the  case  with  2:1  interlace  scanning  in  the  F'l.lR, 
The  only  constraint  is  that  the  NLID  of  the  LED  array  be  approximately 
uniform  from  top  to  bottom.  For  example,  consider  the  thermal  scene  in 
Figure  3a.  Assuming  cold  is  0 and  hot  is  1,  Figure  2b  shows  the  video 
generated  by  the  LED  for  the  even  and  odd  fields.  With  about  400  active 
video  lines  sampled  from  the  vidicon  every  field,  there  are  five  equally 
spaced  sampled  lines  between  each  of  the  80  LEDs.  The  approximate 
video  from  the  vidicon  scanning  in  Figure  3 is  shown  in  Figure  4.  Figure 
4a  shows  the  detector  and  line  scanning  format;  Figure  4b  shows  the  video 
waveforms.  It  can  be  seen  that  synthetic  DC  restoration  is  still  usable  on 
this  video  since  Assumptions  1 to  3 are  still  satisfied. 

In  conclusion,  for  scenes  satisfying  Assumptions  1 to  3 and  with  proper 
vidicon  alignment,  the  synthetic  DC  restoration  method  should  work  well 
with  the  common  module  F'LIR. 


To  give  an  exact  representation  requires  a detailed  study  of  the  FLIR 
optics.  It  is  thought  that  this  is  a fair  representation  of  the  NLID. 
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Figure  4b.  Vidicon  Video  Format 


GLOBAL  GAIN/BIAS  CONTROL 


Global  Gain /Bias  Control 
Implementation  and  Software 

The  algorithm  and  implementation  for  the  PATS  global  gain/bias  control 
approach  were  discussed  beforef  For  review’,  the  global  gain/bias  control 
implementation  diagram  is  shown  in  Figure  5.  The  upper  and  lower  thresh- 
hold  exceedances  of  the  FLIR  output  video  are  counted  during  an  image  field 
and  read  by  an  8748  microprocessor  at  the  end  of  the  field.  From  these 
exceedance  data,  the  gain  and  bias  voltages  are  computed  and  fed  back  to 
the  post-amplifier  stages  in  the  FLIR.  The  algorithm  to  do  the  control  is 
in  software  in  the  microprocessor. 

The  gain/bias  control  algorithm  currently  being  tested  in  8748  softw’are 
fixes  the  bias  voltage  and  varies  only  the  gain.  The  bias  is  fixed  at  the 
midpoint  of  the  upper  and  lower  saturation  regions  of  the  vidicon  and  though 
this  bias  setting  may  not  appear  optimal  when  viewing  the  video  from  the 
FLIR,  it  is  optimal  for  the  vidicon  and  electronics;  image  enhancement  will 
improve  the  resulting  video  signal  for  better  viewing. 

A flowchart  of  the  gain/bias  control  software  is  shown  in  Figure  6.  The 
gain  (G)  is  updated  depending  on  the  relation  of  the  average  number  of 
exceedances  (upper  and  lower  exceedances  summed)  in  eight  fields  of  data 
(N^)  to  three  preset  constants  K^,  Kj,  and  Kg.  These  constants  are 
percentages  of  the  maximum  number  of  exceedances  in  a field  (N),  assuming 
512  samples  per  video  lino.  The  current  values  of  the  constants  are: 


0.002N 


Kj  = O.OIN 
K.,  = 0.03N 


N ?•  20. 000 


I'ho  gain  is  decreased  gradually  by  an  amount  if  exceeds  K.,. 

Decrement  occurs  until  N,^,  falls  below  or  reaches  the  minimum  gain 

(G  . ).  If  N„,  falls  below  the  gain  is  increased  graduallv  bv  an  amount 
mm  1 0 e V . 

AG,  until  N_  goes  abov'e  or  the  maximum  gain  (G  ) is  i-eached.  The 
quantities  AGj^  and  AGj  arc  fixed  with  AG^^  greater  than  AG^,  The  incre- 
ment values  are  expressed  as  a percent  of  the  total  gain  adjustment  range 
and  have  the  following  vi\lues: 

AGp  = 4‘’‘. 

AGj  = 2‘T 

These  small  values  of  AG^^  and  AG^  prevent  oscillation  but  result  in  slow 
adjustment  of  the  gain.  The  adjustment  is  slow  because  the  settling  time  of 
the  FLIR  output  to  a step  change  in  gain  control  v'oltage  is  about  0.  5 seconds, 
or  30  fields.  As  a result  of  this  and  the  sampling  of  eight  frames  of  thresh- 
hold  data,  nearly  0.7  seconds  of  time  is  lost  per  gain  update. 


.•\s  an  example  of  this,  if  the  gain  had  to  be  decreased  by  24  percent,  the 
time  required  to  do  so  would  be  about  4 seconds.  To  skirt  this  problem, 
the  value  of  aGj^  can  be  varied  by  increasing  its  value  when  is  larger. 
Presently,  the  value  of  AG^  must  remain  fixed  because  no  information  is 
available  if  the  scene  falls  below  the  thresholds  (N.^.  zero).  The  addition  of 
scene  standard  deviation  ns  an  adjustment  parameter  for  low  contrast 
scenes  is  being  considered  to  allow  for  a variable  aG^. 
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Global  Gain /Bias  Testi 


The  gain/bias  control  software  is  currently  being  checked  out  and  has  been 
integrated  with  MODFLIR.  It  is  expected  that  the  algorithm  will  work  well 
on  slowly  varying  scenes. 

Modifications  may  be  made  to  include  scene  standard  deviation  as  a contrast 
adjustment  parameter. 

LOCAL  AREA  GAIN/BRIGHTNESS  CONTROL 

The  image  enhancement  local  area  gain/brightness  control  (LAGBC)  algo- 
rithm, implementation,  and  results  on  sample  imagery  were  given  previ- 
ously} 


SECTION  3 


TARGET  SCREENING  ALGORITHM  SIMULATION 


This  section  covers  the  progress  in  this  reporting  period  on  the  target 
scrcener  computer  simulation,  to  include: 

• Data  base  preparation 

• Object  extraction  (segmentation)  examples 

• P’eature  analysis  and  selection 

• Detection  and  recognition  classifier  methodology  and  prototype 
design 

• Performance  of  the  simulation  on  the  test  data  base 

• Interframe  analysis  results 

SUMMARY 

The  Night  Vision  Laboratory  (NVL)  data  base  was  expanded  to  385  digitized 
frames.  Modifications  were  made  to  the  threshold  computations  and  the 
interval  generation  scheme.  A selected  set  of  110  frames  was  processed 
through  the  entire  simuluLian.  The  measured  features  were  evaluated  for 
clutter  discrimination  and  target  classification.  A preliminary  set  of 
features  was  selected  and  used  to  design  a prototype  clutter  rejection 
classifier  and  a classifier  to  discriminate  between  tanks,  APCs,  and  other 
targets.  This  classifier  was  tested  on  the  training  data  and  it  performed 
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satisfactorily.  A set  of  eight  iiulcpoiuk'iU  l-’LIH  frames  from  a stHqueiiee 
(one  second  apart)  with  moving  targets  was  processed  through  this  classi- 
fier and  the  results  are  used  to  illustrate  the  powerful  interframe  analysis 
approach. 

DA’l'A  HASK  PUKPAHA’l'ION 

As  we  reported  in  the  h'irst  Quarterly  Progress  HeportJ  oui-  primary  PA'l'S 
h’l.lH  data  base  is  derived  from  the  h'ort  Polk  I-'IJH  viiieotape  supplied  by 
NVL.  'I'he  "NVl,  imagery"  was  used  because  it  contains  tanks  and  APCs 
at  most  tispcct  ai\gles  ami  reasonable  ranges.  This  day  and  night  imagery 
was  acquired  from  a platform-mounted  KLIH  and  is,  therefore,  appropriate 
for  the  low  elevation  angles  encountereii  in  tank  battle  and  helicopter  pop-up 
mission  scenarios.  In  this  reporting  period  we  digitized  28ri  more  sub- 
frames  of  imagery  from  this  videotape  to  complement  the  NVL  frames  (100) 
digitized  in  the  last  period  in  an  effort  to  complete  representation  of  all 
aspect  angles  of  tanks  and  Al’Cs.  Table  1 summarizes  the  membership 
of  the  NVL  data  base  that  now  contains  ;1HS  digitized  and  annotated  frames. 
As  seen  from  Table  1,  wc  still  do  not  have  a complete  repi'esentation  of  all 
aspects  of  APCs.  This  is  a reflection  of  the  available  aspect  angles  in  the 
viiieotape.  Figure  7 is  a collection  of  sample  frames  from  this  viata  base. 

It  illustrates  the  sensor  and  scene  characteristics  and  shows  how  viivi'rs»> 
target  signatures  can  be  under  different  imaging  conditions. 

IMAGF,  SFGMF.NTA  TlCiN 

A few  changes  were  made  to  the  segmentation  scheme  reported  in  the  PA  I'S 
January  1978  Quarterly  Progress  Heport  and  are  described  below. 
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TABLE  1.  SUMMARY  OF  THE  DIGITIZED  FLIR  DATA  BASE 
(NVL  Data) 


Total  number  of  frames  = 385 

Thresholds 

Instead  of  the  two  thresholded  binary  images--one  Ihresholded  by  a posi- 
tive threshold  and  the  other  by  an  "absolute  value"  threshold- -we  now  have 
two  binary  images,  one  obtained  by  a positive  threshold  and  the  other 
by  a negative  threshold  (about  the  background  estimate).  This  scheme 
extracts  separately  components  which  are  hotter  and  colder  (than  the  back- 
ground). It  is  preferable  to  merge  these  components  later,  following 
syntactic  rules,  than  have  an  absolute  value  threshold,  which  does  not  have 
the  global  information  while  at  the  low  levels  of  the  segmentation  scheme 
and  therefore  tends  to  merge,  for  example,  cold  clutter  with  a hot  target. 

Threshold  Selection 

The  two  intensity  thresholds  (hot  T , and  cold  T ) are  now  computed  using 

11  c 

a recursive  line  average  (i.  e. , the  standard  deviation).  For  line  n, 

Tj^(n)  = a(n)  and  T^(n)  = a (n) 
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Figure  7,  Sample  Digitized  FLIR  Imagery  from  the  N'VL  ’'.'ideotape 


a(n)  is  given  by  a first  order,  recursive  smoothed  estimate  of  the  standard 
deviation  of  previous  lines: 


a (n)  = (1  - y)-^  Z I(n  - 1,  M)  - b(n  - 1,  M)  + Yo(n  - 1) 

M 

I(n-1,M)  is  the  image  intensity  at  the  point  (n-l,IVl),  and  b(n-l,M)  is  the 
background  estimate  at  that  point,  y determines  the  number  of  lines  over 
which  this  averaging  is  done,  y = 0.  9 was  chosen  after  experimentation  and 
gives  an  effective  exponential  weighting  that  decays  to  0.  1 in  22  lines. 
Besides  being  easy  to  implement,  this  way  of  computing  the  threshold  re- 
tains the  advantages  of  a "local"  standard  deviation  estimate  based  only  on 
the  previous  line  and  is  insensitive  to  anomalous  ehanges  in  the  standard 
deviation  from  line  to  line.  Thus,  this  variance  estimate  replaces  the 
global  estimate  reported  in  the  January  1978  PATS  report?  An  identical 
scheme  was  implemented  for  smoothing  the  edge  standard  deviation  estimate 
for  the  edge  threshold  computation. 

Interval  Generation 

The  interval  generation  criterion  reported  in  the  January  1978  PATS  report^ 
is  asymmetrical;  that  is,  it  depends  on  the  order  of  the  data  presentation  to 
the  algorithm  (right  to  left  vs.  left  to  right).  This  is  because  an  interval  is 
started  when  an  edge  and  a bright  sequence  are  encountered;  the  interv'al  is 
terminated  when  the  bright  sequence  is  interrupted  (according  to  a smoothing 
criterion).  Therefore,  if  an  etlge  is  present  at  the  leading  end  of  a bright 
streak,  an  interval  is  generated,  but  if  it  is  at  the  trailing  end,  no  interval 
is  generated.  Figure  8a,  b,  c,  and  d illustrates  this  phenomenon  where  we 
see  that  the  tank  has  pronounced  trailing  edges,  but  the  leading  edges  are 
not  consistent.  This  causes  the  missing  intervals. 
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FLIH  Frame 


H rights 


Intervals 


Figure  }1.  lU'sult  of  One- Directional  Interval  Generation  Criterion 


To  alloviato  this  problem,  a biiUreetioiial,  interval  generatioti  erilt'rion  was 
' conceived,  simulated,  and  has  proven  robust.  With  this  concept , an  interval 

is  started  if  there  is  a streak  of  brights  (or  separatelv,  colds)  and  terminated 

I 

when  the  bright  ceases.  Hut  the  interv”al  is  accepted  if  and  only  if  there  are 
either  leading  or  trailing  edges  at  the  ends  (see  Figure  9).  We  call  this 
valnlation  of  the  inlervail  and  it  is  quite  easy  to  implement.  The  result  is 
identical  to  applying  the  previous  interval  generation  criterion^  twice  to  the 
scan  line  (forward  and  backward).  This  is  shown  in  l’'igure  10,  where  the 
intc'i'wals  are  much  more  consistent  than  in  Figure  Sd. 

Ilesults  of  Segmentation  of  NVL  Data  Hase  I 

^ 

I 

A subset  of  110  images  were  processed  through  the  segmentation  simulation. 

Figure  11a,  b,  c,  d,  e,  and  f shows  various  stages  of  the  segmentation  pro-  i 

cess.  I'igure  11a  is  the  original  daytime  FLIH  frame  containing  an  APC 
amt  three  tanks.  Figures  11b  and  c are  the  "hot"  and  "cold"  thresholded 
binary  images.  Figure  lid  is  the  thresholded  binary  edge  image.  The  hot 
and  cold  binary  images  arc  separately  associated  with  the  edge  image  to 
yield  intervals  (on  a scan  line  basis).  Hot  binarized  points  give  rise  to 
hot  interv^als  and  cold  points  give  rise  to  cold  intervals.  'I'heir  identities 
are  kept  separate  (ITgure  I le).  Hoth  hot  and  cold  intervals  are  then  separ- 
ately associated  into  hot  and  cold  bins  (object  components).  This  is  shown 
in  l-’igurellf.  Here,  the  APC  is  entering  the  field  of  \new  and  the  middle 
tank  is  extracted  as  a hot  component  (the  engine)  and  a cold  component. 

(^nly  the  engines  of  the  other  two  tanks  are  segmented  as  hot  bins. 

More  examples  of  the  object  extraction  process  selected  from  the  110  frames 
processed  appear  in  Figures  12  through  IS. 

I 

i 

i 


23 


RESULT  OF  BIDIRECTIONAL 
INTERVAL  GENERATION 


Figure  9.  Symmetric  Interval  Generation  Criterion 


d.  Binary  edge  image 


e.  Object  intervsils 


TANK 


f.  Extracted  object  components 

Figure  11.  Results  of  a Scene  Adaptive  Segmentation 
Sclieme  (concluded) 
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Figure  14.  Examples  of  Object  Extraction 


Those  110  images  resulted  in  a total  of  1,746  bins  or  extracted  objects. 
Ground  truth  was  subsequently  provided  manually  for  each  object  by  pains- 
taking examination  of  the  segmentation  output,  the  video  disk  image,  and 
the  videotape.  This  resulted  in  393  targets  and  1,353  clutter  objects.  A 
large  number  (810)  of  these  clutter  objects  occurred  because  of  a digitiza- 
tion characteristic  on  either  edge  of  the  image--tlu'  zoi'o  video  portion  was 
digitized  on  some  of  the  frames.  This  gave  rise  to  a number  of  small 
clutter  bins.  This  can  be  seen  in  Figure  llfonthe  sides  of  the  plot. 

Figure  16  is  a dendrogram  that  summarizes  the  ground  truth  on  the  extracted 
objects.  'I'he  numbers  in  parentheses  denote  the  membership  of  each  class. 
Ground  truth  was  separately’  provided  to  hot  and  cold  (relative  to  the  back- 
ground) objects.  "IMT”  in  Figure  16  means  "missegmented"  targets;  in 
our  opinion  these  targets  should  not  be  used  in  training  the  recognition 
classifier  because  they  are  likely  to  be  very  different  from  the  paradigms 
for  the  VTxrious  target  classes.  Targets  which  were  truly  unidentifiable  by 
the  observer  were  labeled  U'l'  (unidentifiable).  We  must  stress  again  that 
the  ground  truth  was  provided  interactively  with  the  help  of  the  videotape 
and  not  just  on  the  basis  of  single  frames.  Therefore,  Uie  number  of 
improperly  segmented  targets  (MTs)  should  be  viewed  in  this  context.  The 
examples  in  Figures  12througli  15,  where  some  of  the  targets  are  barely 
visible  on  the  siitgle  frames,  also  support  this  point.  The  hot  spot  classes 
for  tanks  were  necessitated  because  occasionally  only  the  engines  of  the 
tanks  were  extracted.  When  the  whole  tank  was  extracted,  it  was  placeil 
in  a different  class.  This  was  an  attempt  to  recognize  the  tanks  by  the  hot- 
spots alone  when  only  hotspots  were  visible. 
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FEATURE  EXTRACTION  AND  EVALUATION 


The  PATS  classification  process  proceeds  in  two  cascaded  stages,  (^nce 
the  object  is  extracted,  a decision  is  made  as  to  whether  it  is  a potential 
target.  This  screening  step  is  clutter  discrimination  and  should  be  simple 
and  fast,  using  easy  to  compute  features.  If  an  object  passes  this  test, 
more  measurements  are  made  and  the  object  is  classified  according  to 
target  type  (tank,  APC,  unknown,  etc. ).  This  two-stage  process  weeds 
out  obvious  (and  abundant)  clutter  before  expensive  I’ecognition  feature 
measurements  need  be  made,  thus  eidiancing  the  throughput  of  the  system. 

Table  2 shows  the  featui'es  evaluated  as  candidates  for  the  clutter  I'ejection 
process.  An  initial  pi'ototype,  cluttei'  rejection  classifier  was  designed 
using  these  features.  The  cascaded  threshold  classifier  (CTC)  program, 
which  is  a powerful  analysis  and  design  tool,  w-as  used  to  derive  this  classi- 
fier. Figure  17  illustrates  the  C'I'C  classifier  structure,  which  is  a piece- 
wise linear  classifier  consisting  of  a number  of  linear  discriminants. 
Training  the  classifier  consists  of  detei'mining  the  various  discriminant 
coefficients  that  describe  the  hyperplanes  and  assigning  each  resultant 
region  (formed  by  the  hyperplane)  to  one  of  the  classes  under  consideration. 

Three  stages  comprise  the  clutter  rejection  classifier.  'The  first  stage  is 
a set  of  simple  cascaded  thresholds  on  the  area,  bright  count,  average 
width,  and  average  target  intensity.  This  was  designed  to  remove  the 
"easy"  pathological  clutter  described  previously  (approximately  800  bins). 
'The  second  stage  uses  these  thresholds  and  edge  count,  average  background 
intensity,  average  target  intensity,  average  target  contrast,  amt  perimeter/ 
/area.  This  stage  is  to  divide  the  data  set  into  groups  (subspaccs)  so  that 
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TABLK  2.  CLUTTER  DISCRIMINATKW  FEATURES 


• 

Area 

• Average  target  intensity 

• 

Edge  count 

• Peak  target  intensity 

• 

Edge  discontinuity 

• Length/width 

• 

Bright  count 

• Average  width 

• 

Edge  straightness 

• Average  target  contrast 

• 

Average  background 

• Perimeter  A area 

• 

Minimum  target  intensity 

• Bin  color  (hot /cold) 

within  each  subspacc,  the  targets  and  clutter  are  separable  by  linear  hyper- 
planes,  which  is  done  in  the  third  stage.  We  omit  the  precise  structure  of 
Ibis  classifier  for  the  sake  of  brevity.  The  results  of  this  clutter  rejection 
classifier  are  given  below  in  Table  3. 


TABLE  3.  CLUTTER  REJECTION  PEREORIMANCE 


Reject 

Accept 

Clutter 

1227 

124 

'rarget 

70 

315 
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The  maximum  number  of  vector  products  (discriminant  computations) 
needed  is  a measure  of  the  classifier  computational  complexity.  The  clutter 
rejection  classifier,  being  a multistage  classifier,  poses  different  computa- 
tional requirements  depending  on  which  branch  of  the  classification  tree  one 
goes  down.  However,  the  maximum  number  of  vector  products  needed  for 
the  deepest  branch  is  5,  and  the  maximum  number  of  features  involved  at 
any  given  discriminant  computation  is  10. 

We  have  deliberately  biased  the  clutter  rejection  classifier  in  favor  of 
targets  because  we  do  not  wish  to  miss  targets  at  this  early  stage.  More- 
over, any  accepted  clutter  may  yet  be  removed  in  the  subsequent  (recogni- 
tion and  interframe  analysis)  stages. 

This  classifier  can  be  further  optimized  to  minimize  the  number  of  features 
used  (by  further  feature  evaluation)  and  the  number  of  stages  of  hierarchi- 
cal discrimination.  This  will  be  done  in  the  next  reporting  period.  Further 
simplification  of  the  classifier  has  the  added  benefit  of  making  it  more  robust 
with  respect  to  the  training  data. 

HECCXINITION  CLASSIFIER 

The  candidate  recognition  features--three  classes  of  moment  features  and 
the  Fourier  boundary  descriptors  (FBD)--wero  described  in  the  first  interim 
report?  Each  class  of  these  features  was  evaluated  to  determine  its  discrim- 
inatory power  with  the  well-segmented  target  bins. 
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The  Fourier  boundary  descriptors  proved  to  possess  very  little  separation 
among  the  various  target  classes.  Two-dimensional  scatter  plots  of  pairs 
of  these  features  taken  at  a time  showed  that  all  the  classes  appeared  to 
come  from  the  same  distribution.  Figures  18  and  19  show  scatter  plots  of 
two  pairs  of  the  Fourier  amplitudes  to  illustrate  this  point.  As  a consequence, 
the  FBD  features  were  eliminated  from  further  analysis. 


The  moment  features  proved  to  be  of  greater  value  for  class  discrimination, 
but  there  was  significant  correlation  among  similar  moments  from  each 
class  (boundary,  silhouette,  and  intensity).  Figure  20  shows  an  example  of 
this  correlation  betw'een  (intensity)  and  (silhouette).  Therefore, 
for  the  prototype  classifier  we  chose  the  six  intensity  moments,  the  peak 
and  average  contrasts,  and  the  length /width  (p+o/2)  features  (Table  4).  Because 
all  moments  are  normalized  by  (^20  ^02^  PATS  first  interim 

reportS,  the  normalized  M02  perfectly  negatively  correlated. 

Therefore,  was  omitted. 


A prototype  classifier  was  designed  with  these  nine  features  using  the  cas- 
caded threshold  classifier  programs.  Eight  target  classes  were  defined 
(Table  5).  Linear  discriminants  were  computed  to  separate  all  pairs  of 
classes  (28  pairs).  All  the  target  vectors  were  projected  onto  directions 
orthogonal  to  each  of  these  hyperplanes,  and  these  one-dimensional  projec- 
tions were  analyzed  for  class  separation.  Those  that  yielded  the  maximum 
separation  (or  clustering)  of  the  various  classes  were  chosen  and  thresholds 
were  determined  for  each  projection  from  this  analysis.  A total  of  six 
discriminants  were  chosen. 
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Figure  18.  B’ourier  Amplitude  Features  (a^,  a^) 
(Show  no  separation  between  target 
classes  A,  B,  C,  D) 
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TABLE  4.'  RECtKlNITION  FEATURES  USED  IN 
PROTOTYPE  CLASSIFIER 

Intensity  Moments 
*^11 

^30 

^21 

^•2 

Peak  Contrast 
Average  Contrast 
Leiogth /Width 

TABLE  5.  TARGET  CL^VSS  DEFINITION  FOR  THE 
RECOGNITION  CLASSIFIER 


Class 


1 


Definition 

Cold  Tanks 

^ y 

/ 

Cold  Tanks 

1 

Cold  APCs 

Hot  Spots 

f 

(from  tanks  only) 

Hot  Tanks 

f 

Hot  APCs 

f 

Hot  APCs 

1 

Cold  UTs  and  Hot  UTs 

1 

42 


j 

« 

i 

I 

■ 

b 


i 


' 


0 

These  six  discriminants  define,  at  most,  2 regions  in  the  hyperspace  (see 
Figure  17  for  example).  The  next  step  defines  the  class  membership  of 
each  region  to  determine  how  the  training  data  were  classified.  This  step 
simultaneously  designs  the  classifier  and  yields  an  estimate  of  the  classi- 
fier performance.  This  process  was  done  by  histogramming  the  training 
samples  into  the  various  regions  and  assigning  each  nonempty  region  to  a 
class  depending  on  the  majority  class  of  the  target  training  samples  that 
fell  in  the  region. 

Once  the  classifier  was  designed,  all  nonclutter  samples  (including  the 
missegmented  targets)  were  run  through  the  classifier.  Table  6 summarizes 
the  classification  results  of  the  entire  data  set  for  both  stages  of  classifica- 
tions (clutter  rejection  and  target  classification).  The  number  of  false 
alarms  is  also  shown  (102  in  107  frames)  and  is  within  the  design  goal 
maximum  of  one  per  frame.  As  we  will  see  later,  interframe  analysis 
will  significantly  reduce  this  rate  even  further.  From  this  table,  probability 
of  detection  equals  78  percent.  Among  the  samples  that  were  classified  as 
tanks  or  APCs,  tanks  were  correctly  classified  69  percent  of  the  time. 

The  corresponding  probability  of  correct  recognition  for  APCs  was  91  percent. 

INTERFRAME  ANALYSIS 

To  test  the  validity  of  interframe  analysis  for  target  classification,  a 
sequence  of  FLIR  frames  one-third  second  apart  were  acquired  on  a video 
disk  (Figure  21a).  The  sequence  represents  moving  targets  (APC  and  three 
tanks)  with  partial  and  complete  occlusion  (of  the  middle  tank)  and  partial 
occlusion  of  the  two  tanks  on  the  right  of  the  field  of  view  with  each  other. 

Eight  frames  (now  one  second  apart)  were  selected  from  this  sequence  and 
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independently  processed  through  the  segmentation  and  feature  extraction 
simulation  programs.  Figure  22  shows  a sequence  of  eight  frames  from 
Figures  21a  and  21b  segmented  individually  using  the  current  PATS  simu- 
lation. Note  that  the  APC  on  the  left  is  entering  the  field  of  view  and  is 
completely  extracted  in  Frames  2 and  3.  It  is  only  partially  segmented  in 
Frames  4 and  5 and  reappears  as  complete  in  Frames  6 and  8.  In  some 
frames,  hot  spots  on  the  tanks  are  extracted  because  the  colder  areas  of 
the  tanks  are  lower  contrast  with  respect. to  the  background.  The  colder  , 
areas  of  the  middle  tank  are  also  extracted  in  Frames  1,-  2,  5,  6,  and  8. 

This  tank  is  partially  occluded  behind  a tree  in  Frames  3 and  4 and  com- 

» 

pletely  occluded  in  Frame  5.  This  sequence  also  illustrates  targets  partially 
occluding  one  another  (the  two  tanks  being  merged  in  Frames  6' and  8). 


TABLE  6.  PERFORMANCE  ON  TRAINING  DATA 
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The  feature  vectors  from  these  extracted  objects  were  input  to  the  clutter 
rejection  and  recognition  classifiers.  The  result  of  this  classifier  for  eight 
objects  is  shown  in  Table  7 as  object  classification  sequences.  All  four 
targets  are  represented,  as  are  four  frequently  occurring  clutter  objects. 
Note  that  the  clutter  objects  do  not  always  appear  on  every  frame  :it  the 
same  relative  location,  which  is  an  obvious  cue.  This  sequence  also  illus- 
trates the  concepts  of  decision  smoothing  (the  use  of  maximum  a posteriori 
likelihood  classification  based  on  the  decision  strings,  for  example).  Thus, 
the  "U"  classification  on  Frames  6 and  8 could  be  changed  to  "T"  as  a con- 
sequence of  decision  smoothing.  Similarly  the  decision  "A”  on  object  d in 
Frame  7 would  be  changed  to  a "T"  upon  interframe  analysis. 
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TABLE  7.  OBJEC  r CLASSIFICATION  FOH  SEQUENCE 
FROM  FIGURE  22 


T = Target 
C = Clutter 
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These  rules  for  decision  smoothing  are  based  on  1)  the  statistics  of  the 
scene  (the  a priori  probability  of  tanks.  APCs,  etc.,  in  the  scene);  2)  the 
bias  of  the  classifier  (the  probability  that  a tank  is  classified  as  an  APC 
and  vice  \ersa);  and  3)  the  assigned  priority  of  the  target  class  (how  critical 
it  is  not  to  call  an  APC  a tank).  The  first  and  third  parameters  depend  on 
the  operational  scenario  and  can  be  specified  by  the  tank  commander,  for 
example.  The  second  parameter  could  be  determined  by  processing  sequences 
of  imagery  through  the  simulation. 

However,  we  feel  that  a cost-effective  approach  to  determining  the  classifer 
bias  will  be  to  use  the  PATS  hardware  (upon  completion)  to  gather  statistical 
data  for  the  interframe  decision  smoothing  algorithm.  Processor  2 in  PATS 
is  eminently  suitable  for  this  task  because  it  is  easily  programmable  and  can 
be  interfaced  to  external  peripherals  for  data  transfer. 
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SECTION  4 


SYSTEM  DESIGN 


HAHDWARE  DESIGN 

The  PATS  hardware  can  be  broken  up  into  several  functional  units,  as  shown 
in  Figure  23.  These  functional  blocks,  detailed  later,  are: 

• Autothreshold--The  automatic  adaptation  to  varying  contrast  levels 
that  produce  three  binary  signals--hot,  cold,  and  edge. 

• Interval  gencration--The  logical  combining  of  the  binary  signals 
generated  by  autothreshold  and  a valid  segmented  object  made  up 
of  several  intci’vals. 

• First  level  features--The  real-time  generation  of  specific  values 
associated  with  each  interval. 

• Intensity  data--Thc  storage  of  digitized  video  data  over  the  scene 
of  interest. 

• Computer  system  l--Processes  the  first  level  features  and  intensity 
data  for  the  detection  and  recognition  of  specified  target  types. 

• Computer  system  2--Provides  for  the  diagnostic  checkout  of  the 
system  and  determines  the  type  of  symbol  to  be  generated. 

• Symbol  generation- -The  generation  of  a symbol  specifying  the 
target  type  recognized  and  the  location  in  the  scene.  The  symbol 
is  added  to  the  displayed  video. 
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Functional  Units 


Autothreshold 

I 

i ■ 

^ The  autothreshold  structure  is  shown  in  Figure  24.  It  consists  of  two  differ- 

ent  two-dimens’onal  operations  (edge  filter  and  background  filter).  An 
adaptive  threshold  is  calculated  for  each  scan  line.  The  threshold  calculated 
becomes  the  threshold  for  the  next  scan  line.  The  processed  analog  data  are 
i compared  to  their  respective,  either  a binary  edge  or  a binary  hot  or  cold, 

threshold.  If  the  data  exceed  their  threshold,  then  a logical  threshold  is 
generated.  It  will  remain  one  as  loiig  as  the  data  exceed  the  threshold. 

In  Figure  25,  the  edge  signal  derivation  is  shown.  It  is  two-dimensional  in 
that  three  horizontal  scan  lines  are  used  in  calculating  the  edge.  The  imple- 
mentation shown  is  for  the  horizontal  component  only.  Three  lines  of  video 
are  added  together.  A delayed  version  of  this  sum  is  then  subtracted  to  give 
a positive  voltage  for  a rising  edge  and  a negative  voltage  for  a falling  edge. 
The  absolute  value  is  then  taken,  giving  all  positive  voltage  values.  The 
data  are  compared  to  the  threshold  T . When  edge  is  greater  than  T , 
the  comparator  then  produces  a logic  one  signal. 

The  edge  threshold  is  calculated  as  shown  in  Figure  26.  The  edge  signal  is 
integrated  over  the  line  time  to  get  the  average  edge  signal  during  one  hori- 
zontal TV  line.  During  retrace,  the  input  to  the  integrator  is  grounded. 

The  average  output  is  recursively  filtered  using  the  parameter  "a"  as  the 
parameter  that  determines  how  many  previous  values  are  used.  The  previ- 
ous sampled  and  held  value  is  multiplied  by  "a”  and  the  edge  average  is 
multiplied  by  (1-a).  This  value  is  sampled  during  retrace  and  multiplied  by 
the  constant  K„  which  gives  the  threshold  value  T . 
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Autothreshold  Structure 
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The  background  filter,  shown  in  Figure  27,  is  a two-dimensional  recursive 
filter  which  operates  with  two  time  constants --one  for  the  vertical  direction 

' and  one  for  the  horizontal  direction.  The  filter  that  operates  in  the  horizon- 

1 

j tal  direction  is  a low  pass  RC  filter  whose  time  constant  is  a few  pixels. 

The  vertical  filter  consists  of  a summing  junction  and  a horizontal  line  delay 
for  storage  of  the  background  estimate.  The  input  to  the  background  esti- 

[ 

' mate  is  zero  during  times  of  a potential  object,  as  indicated  by  the  presence 

of  an  interval  on  the  previous  scan  line. 

The  switch  is  needed  because  the  presence  of  a high  contrast  object  would 
get  into  the  background,  and  if  the  object  were  large  enough,  the  background 
estimate  might  equal  the  object.  This  buildup  is  shown  graphically  in  Figure 
28.  The  switch  is  turned  off  when  there  is  enough  difference  between  the 
object  intensity  and  the  background  estimate.  The  switch  is  turned  back  on 
when  there  is  little  difference  between  the  intensity  and  background  estimate. 

The  bright  threshold  used  tor  the  binary  hot  and  binary  cold  derivations  is 
calculated  in  a similar  manner  to  that  for  the  edge  filter.  The  calculation 
implementation  is  shown  in  Figure  29.  The  threshold  is  based  upon  the 
difference  between  the  raw  video  and  the  background  estimate.  There  are, 
however,  separate  multipliers  for  the  hot  and  cold  threshold,  so  that  differ- 
ent thresholds  can  be  used  for  signals  hotter  than  background  and  colder 

? than  background. 
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Interval  Generation 


The  interval  generation  consists  of  starting  and  stopping  criteria  as  well  as 
a validation  criterion.  Shown  in  Figure  30  is  the  overall  interval  generation. 
Inputs  are  the  hot  and  cold  binary  signals,  the  edge  binary  signal,  and  a 
system  clock.  The  output  is  an  interval  and  a validation  pulse. 

The  start/stop  criteria  (Figure  31)  are  based  upon  the  presence  of  a signal 
for  a given  duration  of  time.  The  digital  line  delays  are  used  to  get  proper 
phasing  of  the  edge  and  bright  (hot/cold)  data.  After  the  appropriate  delays 
are  introduced,  a given  M of  N values  must  be  present  before  an  interval  is 
turned  on.  The  same  is  true  for  an  interval  to  be  turned  off. 

The  presence  of  an  edge  must  occur  on  one  of  three  successive  scan  lines 
in  order  to  have  a valid  edge.  This  helps  eliminate  noise  spikes. 

The  validation  criterion  (Figure  32)  consists  of  looking  for  the  coincidence 
of  valid  edges  and  valid  brights  (hot  or  cold).  If  both  occur  and  the  length 
of  the  interval  does  not  exceed  32  clock  pulses,  the  interval  generated  is 
declared  a valid  interval.  A maximum  interval  length  of  32  clock  pulses 
allows  a maximum  target  size  of  1/16  of  the  total  FOV.  When  the  interval 
is  valid,  a pulse  is  generated  which  is  used  to  load  the  first  level  features, 
generated  during  the  interval,  into  appropriate  latches. 
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Figure  30.  Interval  Generation 


Figure  31.  Start/Stop  Criteria 


Validation  Criteria 
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First  Level  Features 

The  first  level  features  generated  during  an  interval  are  shown  in  Figure  33. 
During  one  horizontal  scan  line,  the  maximum  number  of  intervals  that  will 
be  accepted  are  16.  If  the  number  of  intervals  is  greater  than  16,  an  over- 
flow is  generated.  The  interval  signal  enables  various  counters  whose  out- 
put value  is  the  first  level  feature.  These  first  level  features  are: 


Edge  count 

6 bits 

Bright  count 

6 bits 

Hot  or  cold 

1 bit 

Interval  width 

6 bits 

X position 

9 bits 

Background  estimate 

8 bits 

Sum  of  intensities 

8 bits 

The  edge  count  is  a count  of  the  time  duration  the  edge  signal  is  above  the 
threshold  during  the  interval.  The  bright  count  is  the  count  of  either  the  hot 
or  cold  signal  during  the  interval  time.  The  hot  or  cold  bit  is  determined 
by  whether  the  object  is  hotter  or  colder  than  the  background  estimate. 

The  width  count  is  a count  of  the  time  duration  of  the  interval.  It  should 
always  be  greater  than  or  equal  to  the  hot  or  cold  count.  The  X position  is 
the  position  within  the  scan  line  and  is  used  for  matching  successive  inter- 
vals and  for  determining  the  displayed  symbol  position  if  the  object  is  a 
target. 
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INTERVALS 


Level  Features 


The  background  estimate  (Figure  34)  is  a sample  of  the  analog  background 
estimate  near  the  start  of  an  interval.  It  is  digitized  and  then  latched  as  a 
first  level  feature. 

The  sum  of  intensities  feature  is  the  eight  most  significant  bits  of  the  video 
intensity  sum  during  the  interval.  After  the  data  are  latched  and  stored  in 
the  first  level  feature,  the  12-bit  sum  is  cleared  and  waits  for  the  start  of 
the  next  interval.  This  feature  is  shown  in  Figure  35. 

Also  shown  in  Figure  35  is  the  frame  store  or  memory  number  2.  The  data 
are  digitized  in  real  time  and  stored  in  real  time  and  are  used  for  later 
processing.  The  memory  is  512  by  256  by  6 bits  expandable  to  512  by  512  by 
8 bits. 

The  configuration  for  the  memory  (Figure  36)  is  a single-bit  plane  made  up 
of  eight  16K  memory  chips.  This  memory  configuration  allows  use  of 
slower  than  video  rate  memory  chips  but  still  allows  data  to  be  taken  at 
video  rates.  This  process  is  accomplished  by  using  high-speed  shift 
registers  on  the  input  and  output. 

Computer  System  1 

The  first  level  features  and  the  intensity  data  are  processed  by  a single 
high-speed  computer  system.  This  system  (computer  system  1)  is  shown 
in  Figure  37.  The  system  consists  of  a one-line  buffer  (FIFO)  for  the  first 
level  features;  storage  memory  (RAM  1)  for  the  entire  frame-generated 
first  level  features  and  scratch  pad  area;  a 2901-based  CPU;  and  a hardwai'e 
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Figure  34.  Background  Feature 


Figure  36.  Video  Field  Memory  Bit  Plane 


AID 


Figure  37.  Computer  System  1 


multiplier-accumulator.  RAM  2,  already  discussed,  can  also  be  considered 
part  of  the  computer  system.  A direct-memory  access  interface  allows  for 
reading  and/or  writing  into  RAM  1 or  RAM  2 by  a second  CPU. 


The  RAM  1 or  Memory  1 configuration  is  shown  in  Figure  38.  Both  FIFO  mem- 
ories are  configured  64-words  long  by  16-bits.  Since  there  are  48  bits  of  first 
level  features,  the  data  must  be  multiplexed  before  going  into  FIFO  1.  Then  the 
data  are  direct-memory  accessed  into  RAM  1.  At  the  end  of  the  scan  line  all  the 
data  in  FIFO  1 are  transferred  to  FIFO  2.  The  transfer  of  data  into  RAM  1 may 
take  longer  than  the  horizontal  retrace  time  if  the  number  of  intervals  is  quite 
large.  However,  by  doing  a fast  transfer  from  FIFO  1 to  FIFO  2,  an  entire 
scan  line  of  time  is  available  for  transfer. 

Computer  System  2 

After  the  frame  data  are  processed,  target  position  and  class  data  are  trans- 
ferred to  computer  system  2 (Figure  39).  During  operation,  the  data  are  ana- 
lyzed and  a symbol  of  the  proper  size  and  type  is  generated.  The  joystick, 

CRT,  and  floppy  disk  are  used  during  the  diagnostic  checkout  and  training 
phases.  Target  locations  are  identified  for  training  by  the  joystick,  and  the 
statistical  data  for  various  classes  are  stored  on  the  disk. 

Symbol  Generation 


The  symbol  generation  consists  of  writing  to  a one-bit  plane  of  memory  with 
location  synchronized  with  the  video.  The  symbol  generation  scheme  is  shown 
in  Figure  40.  The  symbol  to  be  generated  is  controlled  by  software  and  hence 
can  be  easily  changed  by  changing  a few  locations  in  computer  system  2 
program  memory. 
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Figure  38.  Memory  1 Configuration /interface 


Figure  39.  CPU  2 Configuration 


TIMING  AND  SYNCHRONIZATION 


Associated  with  the  operation  of  PATS  is  a master  timing  and  synchroniza- 
tion scheme  (see  Figure  41).  The  system  must  be  capable  of  working  with 
either  525-  or  875-line  video  data.  The  number  of  samples /line  remains 
fixed  at  910  samples /line.  The  digital  data  will  be  sampled  at  512  samples 
per  horizontal  line.  Also  included  will  be  standard  TV  signal  such  as 
composite  blanking,  vertical  blanking,  and  horizontal  blanking.  A separate 
clock  will  be  provided  for  the  CPU  1 cycle  time. 

PHYSICAL  CONFIGURATION 

It  is  anticipated  that  all  the  PATS  hardware  including  both  CPUs  will  fit  into 
one  ATR  (air  transport  racking)  large  chassis.  Some  of  the  external 
functions  to  be  provided  will  be  test  points,  power  on,  enhancement  on/off, 
and  target  priority  mode  switch.  Conceptually,  this  configuration  will  look 
like  Figure  42.  The  video  level  indicates  the  presence  of  video  at  the  input 
to  the  target  screening  function.  Symbol  control  will  also  be  provided  so 
that  only  a given  cic.ss  will  be  displayed. 

Internal  to  the  chassis  will  be  space  for  28  wired  cards,  though  this  may 
decrease  because  some  will  be  wire  wrap  cards  and  some  will  be  printed 
circuit.  The  approximate  layout  is  shown  in  Figure  43.  Power  supplies 
and  cooling  will  be  an  integral  part  of  the  PATS  chassis. 
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S Physical  Configuration 


SOFTWARE  DESIGN 


Work  has  begun  on  designing  the  implementation  software  for  PATS.  The 
algorithms  that  are  used  in  the  simulation  are  being  put  into  a form  which 
will  run  within  the  system  architecture  proposed  in  the  previous  subsection. 

Figure  44  diagrams  the  proposed  software  functions.  As  was  discussed  in 
the  hardware  subsection,  these  functions  are  partitioned  between  two  CPUs. 

CPU  1 segments  a frame  of  video  data  into  objects,  classifies  these  objects, 
and  then  reports  classification,  size,  and  location  to  CPU  2.  CPU  1 proces- 
ses data  at  the  rate  of  ten  frames  per  second;  that  is,  all  processing  on  a 
single  frame  must  be  completed  within  0. 1 second.  CPU  2 accumulates  data 
passed  to  it  from  CPU  1 over  several  frames,  and  based  on  an  analysis  of 
these  interframe  data,  generates  a graphics  overlay  on  the  operator's  video 
monitor  which  highlights  and  identifies  detected  targets.  In  addition,  CPU  2 
performs  nonreal-time  diagnostics  on  CPU  1 hardware  and  software.  The 
software  functions  of  both  of  these  CPUs  is  discussed  in  more  detail  below. 

Data  Acquisition  ^ 

I 

t 

CPU  1 receives  image  data  via  the  interval  and  intensity  memories  shown  | 

in  Figure  44.  These  memories  buffer  real-time  data  so  that  they  can  be 

1 

processed  by  CPU  1 on  a nonreal-time  basis.  The  autothreshold  hardware  | 

processes  every  sixth  video  field  and  dumps  real-time  interval  data  to  the  | ' 

interval  memory  via  DMA  transfer  on  a scan  line  by  scan  line  basis.  These 
data  are  processed  by  CPU  1,  which  then  stores  the  processed  data  back 
into  the  interval  memory  for  later  use.  It  should  be  noted  that  this  fast 

(under  200  ns  access)  16Kby  16-bit  memory  also  serves  as  a scratch  pad  ^ 
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Figure  44,  Software  Functions 
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for  CPU  1.  The  same  frame  that  is  processed  by  autothreshold  is  also  digi- 
tized and  stored  in  the  intensity  memory.  This  memory  is  accessed  by  CPU 
1 to  obtain  intensity  information  whenever  it  computes  intensity  moments  on 
an  object. 

Interval  data  produced  by  autothreshold  during  a scan  line  are  dumped  to  the 
interval  memory  during  horizontal  retrace  in  the  format  shown  in  Figure  45. 
The  block  of  data  corresponding  to  a given  line  is  headed  by  a packed  word 
giving  the  line  number  and  the  number  of  intervals  which  are  generated  for 
that  line.  Following  this  word  is  a group  of  3 x (number  of  intervals)  words 
containing  the  packed  features  from  each  interval;  i.  e. , three  words  are 
used  for  each  interval.  Currently,  the  features  packed  into  these  words 
are  interval  color  (i.  e. , hot  or  cold--H/C),  starting  X position,  interval 
width,  the  sum  of  intensities  over  the  interval  (n),  background  filter 
sample  (B),  and  interval  bright  and  edge  counts.  No  data  are  placed  in  the 
interval  memory  for  scan  lines  during  which  no  intervals  were  generated. 

A register  maintained  by  the  hardware  indicates  how  much  data  have  been 
loaded  into  the  memory. 

Data  Processing 

Bin  Match]  tig --The  task  of  the  bin  matching  procedure  is  to  associate  the 
one- dimensional  intervals  characterized  by  the  interval  features  into  sets 
of  intervals  which  determine  two-dimensional  objects;  that  is,  CPU  1 
essentially  must  read  interval  data  from  the  memory,  reorder  them,  and 
then  write  them  back.  The  bin  matching  algorithm  has  been  discussed  in  a 
previous  report^  and  will  not  be  described  here  in  detail.  Figure  46  lists 
most  of  its  key  points.  The  algorithm  is  sequential,  so  that  at  any  time 
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Figure  45.  Interval  Data  Format 
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• GROUPS  INTERVALS  INTO  OBJECTS 

• OPERATES  ON  PAIRS  OF  INTERVALS 

• INVOLVES  MIDPOINT  COMPUTATION 
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• OBJECTS  WITH  LESS  THAN  2 INTERVALS  ARE  THROWN  OUT 

• 2901  BASED 

Figure  46.  Bin  Matching  Software 
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during  processing  of  a frame,  a number  of  partially  completed  objects  or 
'bins"  will  exist.  Very  simply,  each  bin  accumulates  object  intervals  on 

] 

successive  scan  lines  that  fall  within  the  midpoint  of  the  previous  interval 
assigned  to  the  bin.  Missed  intervals  are  filled  in  up  to  three  lines.  If 
there  are  no  new  intervals  or  scan  lines,  the  bin  is  considered  closed  and 
further  processing  is  then  done  on  the  bin  (such  as  computing  the  various 
elementary  features). 

Work  is  proceeding  to  develop  an  optimized  version  of  the  bm  matching  algo- 
rithm, which  when  implemented,  will  minimize  the  expected  number  of  com- 
parisons needed  to  match  an  interval  against  a list  of  active  bins.  Bins  will 
be  sorted  in  ascending  order  according  to  the  value  of  the  midpoint  of  the 
last  interval  associated  with  each  bin.  In  this  way,  the  relative  locations  of 
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bins  are  established  across  an  image.  Given  an  interval  to  be  compared 
against  this  ordered  list  of  bins,  the  algorithm  will  then  try  to  establish  the 
necessary  conditions  to  achieve  a bin  match.  Once  these  necessary  condi- 
tions have  been  satisfied  by  some  bin,  sufficient  conditions  are  checked. 

The  necessary  conditions  for  a match  to  occur  are: 

1„  The  bin  and  interval  must  overlap  in  the  X (scan  line)  direction. 

2.  The  bin  and  interval  must  have  the  same  color. 

The  sufficient  conditions  are  the  midpoint  matching  criteria  shown  in  Figure 
46. 

Listing  the  bins  in  ascending  order  will  make  establishment  of  the  first 
necessary  condition  above  very  inexpensive  and  will  make  determination  of 
when  to  stop  comparing  bins  possible  (that  is,  once  a bin  is  found  entirely 
to  the  right  of  an  interval,  it  will  be  impossible  to  obtain  a match  with  that 
bin  and  all  bins  following  it).  Comparisons  to  establish  the  first  necessary 
condition  will  start  with  the  bin  nearest  the  left  edge  of  the  image  and  con- 
tinue with  those  increasingly  toward  the  right.  Bins  which  do  not  satisfy 
the  necessary  conditions  for  matching  an  interval  will  each  require  at  most 
three  comparisons  with  the  interval;  most  will  require  only  one.  Establishing 
the  sufficient  conditions  will  require  four  comparisons.  Therefore,  a bin  that 
matches  an  interval  will  require  seven  comparisons  using  this  scheme.  How- 
ever, most  bins  will  not  even  overlap  the  interval  and  therefore  will  require 
only  one  comparison. 

Figure  47  shows  a map  of  the  way  the  interval  memory  during  bin  matching 
is  expected  to  appear.  Interval  features  are  loaded  sequentially  in  real 
time  into  the  upper  8K  words  of  the  interval  memory  by  the  autothreshold 
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Figure  47.  Map  of  Interval  Memory 


hardware,  using  the  data  format  in  Figure  45.  Two  thousand  words  are 
reserved  for  scratch  storage  required  by  the  bin  matching  procedure.  The 
bins  are  developed  in  the  6K  memory  words  following  this  scratch  area  with 
wrap-around  into  the  upper  8K.  Note  that  bin  matching  essentially  copies 
interval  data  into  the  bins,  so  redundant  interval  data  in  the  upper  8K  of 
memory  can  be  overwritten.  It  is  expected  that  a fixed  length  block  of 
approximately  160  words  will  be  allocated  for  each  bin;  this  block  will 
contain  bookkeeping  data  necessary  while  a bin  is  open  (e.  g. , bin  endpoints, 
midpoint,  missed  scans  count,  etc. ) and  features  computed  over  the  entire 
bin  after  it  is  closed  (e.  g. , total  brights,  moments,  etc.).  Table  8 shows 
a possible  format  for  this  bin  data.  Active  or  open  bins  are  linked  together 
in  ascending  order  using  a linked  list  structure;  once  a bin  is  closed,  it 
is  removed  from  this  list.  Interval  features  that  are  unpacked  to  do  bin 
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TABLE  8.  POSSIBLE  BIN  FORMAT 


Word 

Contents 

r 1 

Address  link  to  next  bin 

2 

Midpoint  (from  last  interval  in  bin) 

3 

Starting  X 

Bookkeeping 

J 

4 

5 

X + width 

Bin  color  (H/C) 

s 

6 

Consecutive  missed  scans  count 

7 

Line  number  on  which  bin  starts 

8 

Total  scans  count  (N) 

9 

Active  scans  count 

X 

11 

Width 

First  interval  { 
in  bln 

[ 13J 

• 

Packed  features 

• 

• 

• 

^4N  + 6 

• 

• 

X 

Last  interval  < 

4N  + 7 

^■M  -1-  H 

Width 

in  bin  1 

4N  + 9 J 

Packed  features 

4N  + 10 

• 

• 

Object  features  (target  contrast. 

moments,  etc. ) 

160 

AO-A060  649  HONEYWELL  INC  MINNEAPOLIS  MINN  SYSTEMS  AND  RESEARCH  —ETC 
PROTOTYPE  AUTOMATIC  TARGET  SCREENER.(U) 

JUN  78  D E SOLANO*  P M NARENDRA*  R C FITCH  DAAK70-77-C 
78SRC54-2 


F/6  17/5 

0246 

NL 


UNCLASSIFIED 


■'I 

. J 


matching  (e.g. , starting  X and  width)  are  stored  unpacked  in  the  bins  to 
avoid  the  overhead  of  having  to  unpack  them  again  later.  Features  which 
are  not  needed  for  bin  matching  are  left  in  their  packed  form.  Note  that 
the  starting  X and  width  of  each  interval  within  a bin  must  be  saved  for  the 
median  filter  and  for  moment  feature  computations. 

Bin  matching  and  loading  of  the  interval  memory  by  autothreshold  will 
proceed  in  parallel  while  the  interval  data  for  a frame  is  being  generated. 
The  interval  memory  will  be  loaded  via  direct-memory  access  transfers 
from  buffers  in  autothreshold.  During  the  time  this  direct-memory  access 
is  taking  place,  CPU  1 will  not  be  able  to  access  the  memory.  However, 
between  these  transfers  CPU  1 will  try  to  do  as  much  processing  as  it  can. 

If  the  frame  being  processed  does  not  produce  many  intervals,  it  is  possible 
that  bin  matching  will  be  complete  by  the  end  of  the  frame.  If  bin  matching 
is  not  completed,  additional  time  will  be  used  to  finish  it. 

Median  Filtering--Once  bin  is  complete,  CPU  1 will  smooth  the  boundaries 
of  each  bin  using  a one- dimensional  median  filter  of  width  three.  The  values 
which  are  input  to  the  filter  are  the  endpoints  of  the  intervals  making  up  each 
bin.  A separate  filtering  operation  is  done  on  the  left- and  right-hajid  edges 
of  each  object  (see  Figure  48).  Each  triplet  of  endpoints  on  a given  edge  is 
sorted,  and  the  middle  endpoint  of  the  triplet  is  assigned  the  middle  value 
from  the  sort.  Note,  however,  that  this  filter  is  nonrecursive;  only 
original  endpoints  are  input  to  the  filter,  not  filtered  ones.  CPU  1 will 
process  all  the  intervals  within  each  bin  and  update,  where  necessary, 

starting  X values  and  widths. 
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SMOOTHS  BOUNDARIES  OF  OBJECTS  FOUND  BY  BIN  MATCHING 


• OPERATOR  ON  INTERVAL  TRIPLETS 

• INVOLVES  SORT  OF  3 VALUES  (2  COMPARISONS)  ON  EACH  EDGE 

LINE  i-1  _| 1 

LINE  i _J  [""1 

LINE  i+1  _) 1 

• 2901  BASED 

Figure  48.  Median  Filter  Software 

Object  Features  and  Classifies — Cbjcct  features  are  computed  by  CPU  1 on 
the  median  filtered  bins  in  a hierarchical  fashion.  That  is,  less  expensive 
features  are  computed  initially  to  do  preliminary  clutter  screening,  and 
more  expensive  features  are  computed  on  the  unrejected  objects  in  order  to 
do  additional  clutter  screening  and  object  recognition.  Table  9 describes 
some  features  that  currently  are  candidates  for  preliminary  clutter 
screening;  Table  10  lists  features  that  currently  arc  being  used  to  do  finer 
target  discrimination.  The  intensity  moments  are  by  far  the  most  expen- 
sive and  are  computed  only  on  those  objects  for  which  it  is  absolutely  neces- 
sary to  do  so.  Clutter  rejection  and  target  classification  is  performed  by  a 
tree-structured  classifier  consisting  of  thresholding  on  simple  features  and 

features  combined  in  discriminant  functions.  The  features  themselves  are 

4 

discussed  in  a previous  interim  report. 

4 

Soland,  pp.  81-100. 
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TAHLK  9.  PHELIMINARY  FEATURES  AND  CLUTTER  PRESCREENING 


• Simple  features  are  computed  for  use  in  early 
clutter  discrimination 

• Four  features  per  object 

Area 

Total  brights 
Average  width 
Average  intensity 

• Involves:  3 additions  (approx.)  per  interval 

2 multiplications,  4 comparisons  per  object 

• Clutter  objects  thrown  out 


As  was  mentioned  previously  in  the  bin  matching  subsection,  object  features 
computed  by  CPU  2 on  the  bins  in  the  interval  memory  will  also  be  stored 
in  these  bins.  CPU  2 is  notified  of  the  location,  size,  and  classification 
of  targets  detected  by  CPU  1.  This  can  be  done  by  having  CPU  1 store 
the  latter  information  in  a fixed  set  of  locations  in  the  interval  memory, 
interrupt  CPU  2,  and  then  allow  CPU  1 to  access  the  information  via 
direct-memory  access. 
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TABLE  10.  PRIMARY  FEATURES  AND  CLASSIFIER 


t t 


Computational  Requirements 

Estimates  of  the  worst  case  computational  requirements  for  CPU  1 arc 
summarized  in  Table  11.  These  estimati' ; were  made  by  first  estimating 
the  amount  of  time  needed  to  perform  arithmetic  operations  (e.  g. , adds, 

[ subtracts,  etc. ) with  all  operands  present,  and  then  doubling  this  time  to 

account  for  memory  accesses  needed  to  fetch  the  operands.  Because 
CPU  1 is  a microprogrammable  processor,  all  estimates  of  computational 
requirements  are  initially  given  in  terms  of  number  of  microinstructions 

or  microcycles  which  are  then  converted  to  nominal  execution  times. 

I 

i I 

I 

I 


• Features  include  those  from  preliminary  screening  plus: 

Total  edges 

Average  background  intensity 
Moments 

Length/average  width 
Average  target  contrast 
Perimeter  7\/Area 
Object  type  (hot  or  cold) 

• Moments  are  by  far  the  most  expensive 

nil,  n‘^0*  ii^l*  n03  centroids  (worst  case) 

• Classifier  does  detailed  clutter  rejection  and  target  recognition 

--  Hierarchical 

--  Discriminant  computations  use  10  features 


TABLE  11.  CPU  1 WORST  CASE  COMPUTATIONAL  REQUIREMENTS 


I 


) 


i'1 

It  was  assumed  that  in  a worst  case  situation,  CPU  1 would  initially  have 
to  process  50  objects,  each  with  a maximum  size  of  32  by  32 pixels.  Bin 
matching  and  median  filtering  would  be  done  on  these  50  objects,  along 
with  the  computation  of  nonmoment  features.  Clutter  screening  by  the 
classifier  would  reduce  this  number  to  ten  32  by  32  objects  on  which 
moment  features  would  be  computed  for  further  clutter  screening  and 
object  recognition. 
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• 

Bin  matching  + median  filter 

50  objects  X 32  intervals /object  x 30  u 1 /interval  = 48K  p | 

( 9.  6 ms) 

• 

Features,  excluding  moments 

50  objects  (each  32  x 32)  x 760  p | /object  = 38K  p | 

( 7.6  ms) 

• 

Moment  features 

10  objects  (each  32  x 32)  x 5.  7K  p ) /object  = 57K  p | 

(11.4  ms) 

• 

Frame  classifier 

10  objects  X 200  p | /object  = 2K  p | 

( 0.  4 ms) 

+ lOOTo  overhead  14jK  p | 

145K  p 1 

( 29. 0 ms) 

( 29.  0 ms) 

290K  p I 

( 58.  0 ms) 

c:>  ~ 60%  LOADING  USING  200  ns  p C\  CLE 

...  

i 

i 

! 

Some  justification  for  the  estimates  in  Table  11  is  offered  below. 

Bin  Matching  and  Median  Filtering — It  is  estimated  that  approximately  30 
microinstructions  will  be  required  for  each  interval  to  do  the  seven  or  so 
comparisons  required  to  match  a bin,  and  the  four  comparisons  required 
to  do  the  median  filter  and  the  bookkeeping  functions  required  in  bin 
I matching  (e.g. , midpoint  updating,  etc.). 

' Features  (excluding  moments)--The  mix  of  arithmetic  operations  on  a 32  by  j 

I 32  pixel  object  is  as  follows  for  various  features: 

f Area  (E  width)  32  adds 


E Edge 
E Bright 

Average  background 

Average  intensity 

Length /average  width 

Average  width 

Average  target  contrast 
2 , 

Perimeter  /area 


32  adds 
32  adds 

32  adds,  1 division 

32  adds,  1 division 

1 division 

1 division 

1 subtraction 

64  adds,  64  subtractions, 

64  absolute  values,  1 division 


Assuming  it  takes  1 microinstruction  to  do  additions  and  subtractions,  3 
microinstructions  to  do  absolute  values,  and  about  64  microinstructions  to 
do  a 16-bit  division,  the  latter  operations  amount  to  about  760  micro- 
instructions per  object. 
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Moments --The  moment  feature  calculations  use  additions,  subtractions, 
multiplications,  divisions,  and  table  look-ups.  In  making  estimates,  it  was 
assumed  that  16-bit  by  16-bit  multiplications  would  be  done  by  using  a 
multiplier/accumulator  chip  external  to  the  CPU  1 arithmetic  logic  unit, 
taking  three  or  four  microinstructions  per  multiplication.  Without  going 
into  a detailed  breakdown  of  the  operations  themselves,  the  estimate  of  the 
number  of  microinstructions  required  to  do  the  arithmetic  operations  for 
the  moments  on  a 32  by  32  pixel  object  breaks  down  as  follows: 


Centroids 

262  microinstructions 

fi02,  p20 

352 

II 

Jill 

2275 

II 

^30,  ^03 

320 

II 

H21.  ^12 

2464 

II 

5673  microinstructions 


or  approximately  5.  7K  microinstructions  per  object. 

Frame  Clnssifier--Assuming  seven  10-feature  discriminant  functions,  each 
involving  10  multiply /accumulates,  the  estimate  is  200  microinstructions 
per  object. 

Summary- -The  totals  are  given  in  Table  11,  including  overhead.  Assuming 
200  ns  per  microinstruction,  the  total  time  to  do  the  calculations  is  estimated 
at  58  ms,  which  is  approximately  60  percent  of  the  available  100  ms  (0. 1 sec). 
Note  that  50  non -overlapping  32  by  32  pixel  objects  cannot  physically  coexist 
in  a frame  using  the  PATS  512  by  256  sampling  grid.  This  fact  tends  to  lend 
some  credibility  to  the  assertion  that  these  are  worst  case  estimates. 
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CPU  2 FUNCTIONS 

The  processing  requirements  of  CPU  2 are  modest  compared  to  those  of 
CPU  1.  Software  functions  for  CPU  2 are  shown  in  Table  12  and  in  the 
block  diagram  in  Figure  44. 

Interframe  analysis  and  symbol  generation  are  the  only  functions  which 
CPU  2 must  perform  in  a field  operational  system.  The  other  functions, 
i.  e. , diagnostics,  feature  dumps,  and  test  data  generation,  are  necessarj' 
for  debuggitig  the  hardware  and  software  in  the  laboratory  or  bench  version. 


T/VBLE  12.  CPU  2 SOFTWARE  FUNCTIONS 


• Memory  diagnostics 

• Feature  dumps  for  off-line  training 

• Interval  features 

• Object  features 

• Test  data  generation 

• Interframe  analysis 

• Symbol  generation 
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1 

. ' The  interframe  analysis  accumulates  the  classification  results  of  at  most 

i 10  objects  from  each  frame  processed  by  CPU  1,  tracks  these  objects  on 

i a frame  to  frame  basis,  and  outputs  a cumulative  classification  for  each  of 

them.  Object  tracking  is  done  only  on  a symbolic  basis;  that  is,  instead 
of  intensity  information,  only  object  class,  size,  and  position  are  used. 
Interframe  analysis  has  been  described  in  a previous  report.^ 

I 

. Symbol  generation  refers  to  the  generation  of  a graphics  overlay  on  the 

P operator's  video  monitor  (see  Table  13).  This  overlay  identifies  and 

highlights  targets  that  were  found  by  CPU  1 and  classified  by  interframe 
analysis.  A dot  graphics  memory  covering  an  entire  frame  is  maintained 
by  CPU  2 and  its  contents  are  continually  read  out  and  mixed  with  the  analog 
video.  CPU  2 maintains  the  graphics  memory  essentially  by  erasing  and 
rewriting  it  every  sixth  frame.  Target  highlighting  techniques  may  include 
drawing  a box  around  each  target  and  generating  some  sort  of  identifying 
symbol;  there  are  several  options  for  highlighting  since  shapes  and  symbols 
will  be  defined  by  software  and  not  hardware. 

TABLE  13.  SYMBOL  GENERATION  SOFTWARE 

• Generates  target  cues  on  video  display 

• X,  Y,  target  type,  target  extents  passed 
from  CPU  1 for  each  object 

• Target  enclosures  generated  by  setting  bits 
in  dot  graphics  memory 
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Diagnostics,  Feature  Dumps,  and  Test  Data  Generation 


These  functions  involve  writing  software  to  interface  with  the  memories  in 
CPU  1,  as  shown  in  Figure  44.  Integrity  of  these  memories  can  be  checked 
by  CPU  2;  features  generated  by  CPU  1 which  are  stored  in  the  interval 
memory  can  be  dumped  to  disk  or  tape  for  checking  CPU  1 software  integrity 
and  off-line  derivation  of  classifier  coefficients.  CPU  2 can  also  generate 
test  data  for  CPU  1 and  store  them  in  the  interval  and  intensity  memories. 


A detailed  test  of  the  PATS  software  can  be  achieved  as  follows.  A frame 


is  processed  using  a version  of  the  PATS  simulation  designed  to  match  the 
implementation  software.  All  data  generated  by  the  simulation  are  saved, 
including  interval  data,  object  features,  and  classification  results.  CPU  2 
is  used  to  load  the  interval  memory  with  the  interval  data,  and  the  intensity 
memory  with  the  digitized  frame.  Next,  CPU  1 is  run  on  the  data  stored  in 
the  interval  and  intensity  memories  (the  autothreshold  hardware  is  not 
used);  CPU  2 is  used  to  dump  the  results  generated  by  CPU  1 for  compari- 
son with  the  simulation  results.  In  another  mode,  CPU  2 functions  can  be 
checked  by  using  diagnostics  internal  to  CPU  2 and  by  allowing  CPU  2 to 
generate  its  graphics  overlay  on  a video  disk  playback  of  the  frame  processed 
by  the  simulation. 


CPU  1 is  a special  purpose  high-speed  microprogrammed  processor  designed 
to  meet  a required  processing  rate  of  10-video  fields  per  second.  CPU  2, 
on  the  other  hand,  is  a more  general  purpose,  off-the-shelf,  "macro- 
programmed"  processor  with  far  less  computational  loading  than  CPU  1.  A 
special  requirement  of  CPU  2 is  that  it  interface  with  peripheral  devices  that 
provide  off-line  storage  (e.  g. , floppy  disk),  character  data  input  (e.  g. , CRT), 
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and  hardcopy  outpu*  (e.  g.,  line  printer).  This  is  because  in  addition 
to  providing  operational  functions  (interframe  analysis,  symbol  generation), 

CPU  2 also  supports  diagnostics  which  provide  a window  into  the  operational 

software  and  hardware.  j 

i 

i 

CPU  1 Software  Development  | 

CPU  1 will  be  programmed  at  the  microinstruction  level,  and  as  such  | 

requires  special  development  tools.  A microassembler  and  text  editor  j 

will  be  used  to  allow  microcoding  at  the  mnemonic  rather  than  the  bit  level.  | 

j 

Frequent  changes  will  be  made  to  the  microcode  during  the  development  ! 

period,  so  during  that  time,  the  microcode  will  reside  in  a fast  RAM  or  j 

writable  control  store  (WCS);  PROM  will  be  used  in  the  operational  system. 

The  microcode  development  facility  will  provide  an  interface  with  the 

WCS  and  will  allow  single-step  execution  and  address  tracing  of  microcode  i i 

during  execution.  i 

I 

CPU  2 Software  Development 

CPU  2 will  be  programmed  like  a typical  minicomputer,  i.  e. , using  an 
assembler  and,  if  available,  a higher  order  language.  The  interframe 
analysis  simulation  is  coded  in  FORTRAN,  so  using'FORTRAN  will  speed 
development  of  interframe  analysis  implementation  software.  Because 
CPU  2 will  be  interfaced  with  sophistieated  peripherals  (e.g. , floppy  disk), 
it  would  be  very  helpful  if  I/O  drivers  for  these  peripherals  were  provided 
by  the  manufacturer  in  a real-time  operating  system.  The  software 
development  tools  for  CPU  2,  unlike  CPU  1.  can  actuallj-  be  resident  on 
CPU  2,  thereby  removing  the  problem  of  loading  the  CPU  2 memory  with 
program  code. 


SECTION  V 
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f PLANS  FOR  THE  NEXT  REPORTING  PERIOD 

F 


During  the  next  reporting  period,  further  refinements  of  the  target  screening 
algorithms  are  planned.  These  refinements  include  improvements  in  1) 
segmentation,  2)  clutter  rejection,  and  3)  recognition  algorithms.  Improved 
segmentation  is  desirable  to  provide  a better  outline  of  the  target  for 
moment  calculations.  In  particular,  a merging  of  hot  and  cold  bins  corre- 
sponding to  different  parts  of  the  same  target  is  necessary.  We  intend 
to  refine  the  feature  selection  for  the  clutter  rejection  classifier  to  provide 
a simpler  and  more  robust  clutter  filter.  We  also  intend  to  simplify  the 
recognition  classifier  by  further  feature  selection  experiments.  Also, 
the  k-nearest  neighbor  classifier  for  recognition  will  be  investigated.  The 
use  of  k-nearest  neighbor  will  simplify  retraining  of  the  classifier  with 
new  training  samples. 

Functional  design  specifications  will  be  written  for  each  of  the  hardware 
and  software  system  elements.  We  also  will  begin  detailed  design  of 
the  timing  and  synchronization  circuits,  CPU  1 and  its  associated  memory  1 
and  memory  2. 

Checkout  of  the  image  enhancement  breadboards  with  the  MODFLIR  will 
also  be  completed  during  the  next  reporting  period. 
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